Object-based attentional expectancies in virtual reality

Modern virtual reality (VR) technology has the promise to enable neuroscientists and psychologists to conduct ecologically valid experiments, while maintaining precise experimental control. However, in recent studies, game engines like Unreal Engine or Unity, are used for stimulus creation and data collection. Yet game engines do not provide the underlying architecture to measure the time of stimulus events and behavioral input with the accuracy or precision required by many experiments. Furthermore, it is currently not well understood, if VR and the underlying technology engages the same cognitive processes as a comparable real-world situation. Similarly, not much is known, if experimental findings obtained in a standard monitor-based experiment, are comparable to those obtained in VR by using a head-mounted display (HMD) or if the different stimulus devices also engage different cognitive processes. The aim of my thesis was to investigate if modern HMDs affect the early processing of basic visual features differently than a standard computer monitor. In the first project (chapter 1), I developed a new behavioral paradigm, to investigate how prediction errors of basic object features are processed. In a series of four experiments, the results consistently indicated that simultaneous prediction errors for unexpected colors and orientations are processed independently on an early level of processing, before object binding comes into play. My second project (chapter 2) examined the accuracy and precision of stimulus timing and reaction time measurements, when using Unreal Engine 4 (UE4) in combination with a modern HMD system. My results demonstrate that stimulus durations can be defined and controlled with high precision and accuracy. However, reaction time measurements turned out to be highly imprecise and inaccurate, when using UE4’s standard application programming interface (API). Instead, I proposed a new software-based approach to circumvent these limitations. Timings benchmarks confirmed that the method can measure reaction times with a precision and accuracy in the millisecond range. In the third project (chapter 3), I directly compared the task performance in the paradigm developed in chapter 1 between the original experimental setup and a virtual reality simulation of this experiment. To establish two identical experimental setups, I recreated the entire physical environment in which the experiments took place within VR and blended the virtual replica over the physical lab. As a result, the virtual environment (VE) corresponded not only visually with the physical laboratory but also provided accurate sensory properties of other modalities, such as haptic or acoustic feedback. The results showed a comparable task performance in both the non-VR and the VR experiments, suggesting that modern HMDs do not affect early processing of basic visual features differently than a typical computer monitor

Knowledge-based interoperability for mathematical software systems

Funding: OpenDreamKit Horizon 2020 European Research Infrastructures project (#676541) and DFG project RA-18723-1 OAF.There is a large ecosystem of mathematical software systems. Individually, these are optimized for particular domains and functionalities, and together they cover many needs of practical and theoretical mathematics. However, each system specializes on one area, and it remains very difficult to solve problems that need to involve multiple systems. Some integrations exist, but the are ad-hoc and have scalability and maintainability issues. In particular, there is not yet an interoperability layer that combines the various systems into a virtual research environment (VRE) for mathematics. The OpenDreamKit project aims at building a toolkit for such VREs. It suggests using a central system-agnostic formalization of mathematics (Math-in-the-Middle, MitM) as the needed interoperability layer. In this paper, we conduct the first major case study that instantiates the MitM paradigm for a concrete domain as well as a concrete set of systems. Specifically, we integrate GAP, Sage, and Singular to perform computation in group and ring theory. Our work involves massive practical efforts, including a novel formalization of computational group theory, improvements to the involved software systems, and a novel mediating system that sits at the center of a star-shaped integration layout between mathematical software systems.Postprin

Accuracy and precision of stimulus timing and reaction times with Unreal Engine and SteamVR

The increasing interest in Virtual Reality (VR) as a tool for neuroscientific research contrasts with the current lack of established toolboxes and standards. In several recent studies, game engines like Unity or Unreal Engine were used. It remains to be tested whether these software packages provide sufficiently precise and accurate stimulus timing and time measurements that allow inferring ongoing mental and neural processes. We here investigated the precision and accuracy of the timing mechanisms of Unreal Engine 4 and SteamVR in combination with the HTC Vive VR system. In a first experiment, objective external measures revealed that stimulus durations were highly accurate. In contrast, in a second experiment, the assessment of the precision of built-in timing procedures revealed highly variable reaction time measurements and inaccurate determination of stimulus onsets. Hence, we developed a new software-based method that allows precise and accurate reaction time measurements with Unreal Engine and SteamVR. Instead of using the standard timing procedures implemented within Unreal Engine, time acquisition was outsourced to a background application. Timing benchmarks revealed that the newly developed method allows reaction time measurements with a precision and accuracy in the millisecond range. Overall, the present results indicate that the HTC Vive together with Unreal Engine and SteamVR can achieve high levels of precision and accuracy both concerning stimulus duration and critical time measurements. The latter can be achieved using a newly developed routine that allows not only accurate reaction time measures but also provides precise timing parameters that can be used in combination with time-sensitive functional measures such as electroencephalography (EEG) or transcranial magnetic stimulation (TMS)

Combined expectancies: the role of expectations for the coding of salient bottom-up signals

The visual system forms predictions about upcoming visual features based on previous visual experiences. Such predictions impact on current perception, so that expected stimuli can be detected faster and with higher accuracy. A key question is how these predictions are formed and on which levels of processing they arise. Particularly, predictions could be formed on early levels of processing, where visual features are represented separately, or might require higher levels of processing, with predictions formed based on full object representations that involve combinations of visual features. In four experiments, the present study investigated whether the visual system forms joint prediction errors or whether expectations about different visual features such as color and orientation are formed independently. The first experiment revealed that task-irrelevant and implicitly learned expectations were formed independently when the features were separately bound to different objects. In a second experiment, no evidence for a mutual influence of both types of task-irrelevant and implicitly formed feature expectations was observed, although both visual features were assigned to the same objects. A third experiment confirmed the findings of the previous experiments for explicitly rather than implicitly formed expectations. Finally, no evidence for a mutual influence of different feature expectations was observed when features were assigned to a single centrally presented object. Overall, the present results do not support the view that object feature binding generates joint feature-based expectancies of different object features. Rather, the results suggest that expectations for color and orientation are processed and resolved independently at the feature level

An fMRI study into emotional processing in Parkinson's disease: Does increased medial prefrontal activation compensate for striatal dysfunction?

Background Apart from a progressive decline of motor functions, Parkinson's disease (PD) is also characterized by non-motor symptoms, including disturbed processing of emotions. This study aims at assessing emotional processing and its neurobiological correlates in PD with the focus on how medicated Parkinson patients may achieve normal emotional responsiveness despite basal ganglia dysfunction. Methods Nineteen medicated patients with mild to moderate PD (without dementia or depression) and 19 matched healthy controls passively viewed positive, negative, and neutral pictures in an event-related blood oxygen level-dependent functional magnetic resonance imaging study (BOLD-fMRI). Individual subjective ratings of valence and arousal levels for these pictures were obtained right after the scanning. Results Parkinson patients showed similar valence and arousal ratings as controls, denoting intact emotional processing at the behavioral level. Yet, Parkinson patients showed decreased bilateral putaminal activation and increased activation in the right dorsomedial prefrontal cortex (PFC), compared to controls, both most pronounced for highly arousing emotional stimuli. Conclusions Our findings revealed for the first time a possible compensatory neural mechanism in Parkinson patients during emotional processing. The increased medial PFC activity may have modulated emotional responsiveness in patients via top-down cognitive control, therewith restoring emotional processing at the behavioral level, despite striatal dysfunction. These results may impact upon current treatment strategies of affective disorders in PD as patients may benefit from this intact or even compensatory influence of prefrontal areas when therapeutic strategies are applied that rely on cognitive control to modulate disturbed processing of emotions

Results of the ROI analysis on the processing of low and high arousing stimuli in Parkinson patients and controls.

<p>Contrasting healthy controls (HCs) with Parkinson patients (PD) showed a differential increase of BOLD signal for HCs in the left (posterior) putamen for low arousal (<b>A</b>) and bilaterally in the putamen for high arousal (<b>B</b>, all <i>P</i>_SVC<0.05). Contrasting Parkinson patients with healthy controls revealed an increased BOLD signal for Parkinson patients in the right dorsomedial prefrontal cortex (dmPFC) for low (C) arousal and high arousal (D, all <i>P</i>_SVC<0.05).</p